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ABSTRACT 


Cognitive and Affective Outcomes: Their Relationship to 
Effective Teaching and Students' Evaluations of Instruction 


OVERALL, JESSE U. 
University of California, Los Angeles 


MARSH, HERBERT W. 
University of Southern California 

An investigation into the relationship between students’ 
evaluations and both cognitive and affective consequences of 
instruction showed positive, statistically significant 
validity coefficients. Cognitive outcomes (students' achieve- 
ment on an end-of-course examination) showed the strongest 
relationships with the CONCERN and EXAMINATIONS rating factors. 
Affective outcomes (students' feelings of course mastery and 
disposition to pyrsue the subject further) showed the strongest 
velationahine Slbh the CONCERN, EXAMINATIONS, INTERACTION, 
and LEARNING rating factors. In terms of the summary rating 
items; 1) students' achievement correlated .41 (p<.05) with 
the Overall Instructor item, but .19 (n.s.) with the Overall 
Course item; and, 2) the affective outcomes correlated between 
-52 and 63 (pe.01) with the Overall Course item, but between 
.22 and .31 (n.s.) with the Overall Instructor item. These 
ndings pioubde further support for the validity of students' 
evaluations, and suggest that by using criteria related to 
both cognitive and affective outcomes, it is possible to 


obtain a much broader insimht into teaching effectiveness. 


COGNITIVE AND AFFECTIVE OUTCOMES: THEIR RELATIONSHIP 
TO EFFECTIVE TEACHING AND STUDENTS’ EVALUATIONS = 


The use of students’ evaluations of instruction as an 
index of effective teaching is still one of the most contro- 
versial issues facing the academic community today. A significant 
aspect of this controversy is the insistance by critics that 
positive relationships be shown between such evaluations and 
generally acceptable validity criteria. 

In the past few years, the most widely emphasized and 
accented index of validity has been that of students' achieve- 
ment. If instructors do not receive the highest ratings from 
those students who learn the most, the use of such ratings 
as an indication of effective teachinp is suspect. 

Recently, the imnortance of affective outcomes and their 
relationship to effective teaching has also generated a preat 
deal of interest. Two primary objectives of any wuerieulian 
are the provision of opportunities to develon 1) knowledge that 
will needs a basis for confidence in subject matter mastery, 
and 2) interest in pursuing, the subject further independently. 
Finding positive relationships between such outcomes and 
students’ evaluations of instructors would provide additional 


support for the validity of students’ evaluations. 
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In light of the apparent importance that cognitive and 
affective consequences have for students in the instructional 
setting, and because of the absence of validity research, par- 
ticularly with respect to affective outcomes, the present study 
was conducted. The purpose of this research then, was to in- 
vestigate--at the college and university level--the relationship 
between students' evaluations of instructional effectiveness and 


both cognitive and affective outcomes of instruction. 


LITERATURE REVIEW 

Cognitive Outcomes 

The most commonly accepted validity index for students’ 
evaluations has been students' achievement. Some researchers 
have focused on the use of actual or expected grades to indicate 
such achievement. Results of these studies have been compre- 
hensively reviewed elsewhere (Costin, Greenough and Menges, 
1971; Trent and Cohen, 1973). However, because grades may be 
biased by subjective factors such as personality or forced grade 
distributions, and because there is some evidence that students 
may award higher ratings if their grading expectations are met 
or exceeded (Marsh, Overall and Thomas, 1976), some measure of 
a student's actual gain in knowledge or understanding is probably 
a more preferable criteria. 

More recent research has utilized students' performance on 
a common examination as its criteria for validity. If students’ 
evaluations of instruction are shown to be positively related 


to students’ performance or achievement, then one may reascnably 
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conclude that these evaluations are valid indicators of instruc- 
tional effectiveness. College and university level studies of 
this relationship have been conducted by Bendig (1953); Centra 
(1977); Cohen and Berger (1970); Frey (1973); Gessner (1973); 
Marsh, Fleiner and Thomas (1975); McKeachie, Lin and Mann (1971); 
Morsh, Burgess and Smith (1956); Overall (1977); and, Rodin and 
Rodin (1972). All studies used a multi-sectioned approach, and 
an emphasis on common content or course examinations was apparent 


in all but the McKeachie study. Although the Kodin and Rodin 


research was the only one to renort negative results, the limitations 


peculiar to that study have been thoroughly discussed by Frey and 
Gessner in subsequent articles. 
Affective Outcomes 

The development of positive attitudes toward course exper- 
iences and subject matter by students has received a great deal 
of attention from educational practitioners in recent years. This 
has resulted in instructor concerns that students feel confident 
about and disposed toward using what they have learned in a course 
after its completion. It is expected that the development of 
positive attitudes toward a subject will not only assist the 
student in mastering the cognitive aspects of the particular 
subject, but will also reinforce these positive attitudes in the 
student after a particular course has ended (Krathwohl, Bloom and 
Masia, 1964; Popham and Baker, 1979). 

Fletcher (1959) underscored the importance of affective 


development when he summarized the significant role reinforcement 
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plays in improving a skill or increasing knowledge. He observed 
that students who developed positive attitudes toward the subject 
matter in a course would voluntarily seek out situations to apply 
the cognitive knowledge they had gained after the course was over, 
thus reinforcing any learning that had previously taken place, 
Weinstein and Fantini (1970) also have pointed out the relation- 
ship between cognitive and affective development. To them, there 
is little chance that subject matter mastery will influence be- 
havior unless it is somehow related to an affective state in the 
learner. 

Pohlmann (1975) sought to determine the relationship between 
specific predictor items and an overall summary item ("in general, 
taught the class effectively") used in his Instructional Improve- 
ment Questionnaire at Southern Illinois University. Using partial 
correlation to statistically remove effects from the other rating 
items, he found that of the 29 predictor items, I] had partial 
correlations of .10 or higher with the criterion; however, only 
three items correlated .21 higher, and one of these was the 
affective item "increased your appreciation of the subject." 

Overall and Marsh (1977) provided summaries of mid-quarter 
(MQ) evaluations by students to a random half of the instructors 
teaching an introductory course in computer programming. The 
993 University of California undergraduates involved were 
typically social science majors who took the course during one 


of the three quarters to fill departmental requirements. The 
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impact of feedback from the students' evaluations was assessed 

by comparing responses of students whose instructors either did 

or did not receive such feedback. Based on end-of-quarter 
evaluations, students of instructors who received MQ feedback 

gave more favorable responses to each of five affective conse- 
quence items (identical to those used in this study) than did 
students whose instructors did not receive feedback. On three 

of the five items, the differences were statistically significant, 
MQ feedback was thus found to be positively related to differences 


in students' affective outcomes, 


METHODOLOGY 


Subjects for the study were 30 sections (924 students) of 
University of California, Los Angeles, undergraduates who completed 
Engineering 10 (E 10), an introductory computer programming class, 
during one of the three quarters in a recent academic year. The 
course was taught by graduate teaching assistants (TAs); the 
syllabus, text and final examination were standardized under 
supervision of the course director. Participating students 
were all non-science majors; a majority came from the social 
sciences, The section was chosen as the unit of analysis. 

At the start of each quarter, students voluntarily selected 
themselves into one of the available sections of E 10, solely on 
the basis of the time a particular section was offered. This 


voluntary selection procedure was crucial in creating a sample 


that approximated a truly random one. A previously validated 
pretest was administered to determine if sections differed to 


any statistically significant extent with respect to ability. 


No such differences were found. Responses to a 38-item 


evaluation questionnaire provide students' perceptions of 


instructional effectiveness at the end of each quarter. 


The evaluation instrument was a variant of the standard form 


developed by the UCLA Evaluation of Instruction Program. A 


&) DIFFICULTY--magnitude of course workload; difficulty of the 
course compared to other courses; amount of time spent on course 
outside of class. Overall instructor and course rating items 
were also included. Factor scores were generated for each 
individual factor “(evaluation dimension). These scores were 
computed by taking an unweighted average of the standardized 
mean of individual responses to the rating items loading highest 


on each factor, These factor scores were then standardized to 


correlation matrix was computed for the items based on individual 
student responses. Factor analysis--a principal components 
solution followed by a varimax rotation--of the correlation 
matrix yielded the orthogonal solution shown in Appendix 1. The 
factors and sample items were as follows: a) Instructor 
CONCERN--presentations made subject more understandable; impli- 
cations, applications and concern that students learn and under- 
stand subject matter were shown. b) BREADTH--various points of 
view were discussed; implications of various theories were 
contrasted, c) ORGANIZATION--course materials and objectives 
were clearly outlined; class presentations were well prepared. 
d) INTERACTION--students were Aieoubadad to ask meaning ful 
questions and were free to disagree with the instructor. 

e) LEARNING--students learned something of value and developed 
an understanding of the implications of the course material; 
intellectual eurtnadty was Stimulated. f) BXAMINATIONS--crading 
was fair and objective; graded materials fairly measured 


knowledge of course as emphasized by the instructor, 


have a mean of 100 and a standard deviation of 10. 
The evaluation instrument also included five items designed 
to provide information about a student's perception of his or 


her affective growth, 


1. Extent...you feel capable of writing and running a 
computer program to solve some problem which you 
may run into in the future, 

2. Extent...you have gained enough understanding of 
what a computer is capable of to be uaeiul to you 
in the future. 

3. Extent...you plan to become (or remain) a member 
of the UCLA Computer Club or find some other source 
which will provide computer time in the future. 

4. Extent...you plan to make practical application 
of the computer in the future. 

5. Extent...you plan to take more computer courses 
(for credit or through the Computer Club) in the 


near future. 
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RESULTS 


The CONCERN and the EXAMINATIONS dimensions had statistically 
significant relationships with cognitive and affective outcomes. 
However, while the correlation between the CONCERN factor and 
achievement was significant, the correlations between CONCERN 
and the five affective variables reached significance on only 
two items. The magnitude of the correlations between the EXAM- 
INATIONS factor, and both students' cognitive and affective 
development was substantial. 

Two evaluation dimensions had statistically significant 
correlations with the affective outcomes, but not with the 
cognitive outcomes. One of these dimensions, LEARNING, had a 
small positive relationship with students' performance on the 
common final examination, but was clearly much more related to 
students’ feelings of course mastery and disposition to pursue 
the subject further. The other dimension, INTERACTION, had 
significant correlations with two of the affective development 
items, and a positive but small relationship with achievement. 

Perhaps the most interesting finding was that the Overall 
Instructor rating item had a statistically significant relation- 
ship with students' achievement, but not with students' affective 
development. On the other hand, the Overall Course rating item 
had a statistically significant relationship with the affective 
outcome variables, but not with stucents' achievement. 


A summary of these results is presented in Table One, 


~----Insert Table One About Here----- 


11 ; 


DISCUSSION 


Cognitive Outcomes 
The results obtained in this study confirmed that by the 


end of the quarter, students' evaluations of instructional 
effectiveness were positively related to their achievement in 
the course. Ratings on the evaluation dimensions CONCERN and 
EXAMINATIONS, as well as on the overall instructor rating item, 
bore the strongest relationships with students' positive 
cognitive outcomes. 

Cognitive development was very definitely related to the 
CONCERN dimension, which includea items reflecting students' 
perceptions of instructors’ concern, enthusiasm, and attempts 
to make the course relevant and understandable. Certainly, one 
way to communicate an interest in something is to be enthusiastic 
about it. Furthermore, if one wants individuals to learn some- 
thing, one can provide positive reinforcement to them by showing 
that their learning is relevant and important--both to them and 
to others. 

The EXAMINATIONS dimension also showed a strong and statis- 
tically significant correlation with students' cognitive develop- 
ment. Those students who perceived a sense of fairness and 
objectivity in the grading and testing processes also achieved 
higher scores on the common final examination. The specification 
of what the student is expected to learn, and the evaluation of 
the student with respect to the material specified, is very 


important to the creation of an effective learning environment. 


Le 
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In this study, the extent to which students perceived they were 
being evaluated on what they were asked to learn was very clearly 
related to their achievement. 

Finally, the Overall Instructor rating showed a strong and 
statistically significant correlation with students' achievement. 
The magnitude of this finding was, of course, quite relevant to 
the question of validity. If ecudenta® summary evaluations of 
instruction showed little relationship to cognitive development, 
then the validity of such evaluations would be suspect. However, 
such was not the case, and the two variables--Overall Instructor 


rating and cognitive development--were clearly related. 


Affective Outcomes 

Definite, statistically significant relationships with 
affective outcomes were found both for the évernt Course 
summary item, and for the rating components of CONCERN, EXAMIN- 
ATIONS, INTERACTION, and LEARNING. There was a positive but 
weak relationship between the Overall Instructor rating and 
affective development. 

Two of five affective outcomes had significant relationships 
with the CONCERN and INTERACTION dimensions, Although not as 
high as the correlations with achievement, the correlations with 
these factors nevertheless indicated a consistent positive relation- 
ship. Thus, an instructor's enthusiasm, attempts to show implications 
and applications of the material, projection of concern that students 
learned and understood the subject, and efforts to foster instructor- 
student interaction were clearly associated with students’ feelings 


of course mastery and future plans for pursuing the subject. 
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Affective development was also associated with the EXAMINATIONS 

factor. Students’ perceptions of a sense of objectivity and fair-» 
ness in their instructor's grading and examination practices 
correlated strongly with feelings of subject matter mastery and 
expectations of pursuing the subject after the end of the course. 
Apparently, students of instructors who both let them know what 
was expected and subsequently evaluated them on the basis of this 
expectation, developed very positive affective relationships to 
the subject matter. The instructor's evaluation procedures were 
thus a positive influence on students' affective development. 

One explanation for this relationship could be that if there 
was effective communication between student and instructor con- 
cerning what was to be learned from the course, and if examinations 
reflected the material emphasized, then students' expectations 
of content mastery were in harmony with actual results. This 
harmony created a reinforcing effect on the students with respect 
to their confidence both in immediate mastery and in future 
probability of success in pursuing the subject independently. 

It is natural for one to pursue a subject for which he or she 
appears to have an affinity, and the students in this study were 


apparently no exception. In addition, given that the EXAMINATIONS 


dimension was also correlated with achievement to a statistically 


significant extent, this explanation is quite plausible. 
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Affective development showed the highest correlations with 
the LEARNING factor. Students' sense of course mastery and 
interest in pursuing the subject matter further after the 
course had ended correlated to.a statistically significant 
extent with students' feelings that they had learned something 
of value, that they understood the implications of the material, 
and that their intellectual curiosity had been stimulated. 

These findings were anticipated because individuals are more 
likely to pursue subjects that they understand, and that appear 
interesting and of value to them. 

Finally, positive affective outcomes had a significant 
relationshp with the Overall Course--but not the Overall Instructor-- 
rating. Those students who, by the end of the quarter, felt both 
a greater sense of course mastery, and a greater interest in study- 
ing the subject matter further after the end of the course, rated 
it higher than those who did not. This. result is intriguing 
because in a previous study using a similar research setting, 
Marsh, Fleiner and Thomas (1975) reported finding a correlation 
of .42 (p¢.05) between students' achievement and Overall Instructor 
rating. However, they also found a positive but statistically 
insignificant correlation of .30 between achievement and Overall 
Course rating. With the addition of affective outcome measures 
in the present study, a possible explanation for such findings 
can be proposed. Apparently, Overall Instructor ratings, while 
positively related to both cognitive and affective outcomes, have 
a nack greater association with students' perceptions of behaviors 


and actions over which the instructor has direct control. Some 
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examples are instructors' presentations that are enthusiastic 

and understandable, and grading that was fair and objective. . 
The Overall Course rating--while positively related to both 
cognitive and affective outcomes--had a much greater association 
with students' feelings of course mastery and interest in studying 
the subject after the end of the course. Thus, while students’ 
assesments of their instructors reflected their level of cognitive 
development as a result of the course, students' assessments of 
their course reflected their level of affective development. 
Because instructors seek to insure that their students attain 

both cognitive and affective outcomes in most teaching situations, 
then the extent to which these outcomes have been attained can 

be determined from students' evaluations. 

In conclusion, it should be noted that not all evaluation 
factors or items reflected both cognitive and affective outcomes. 
Previous research has identified those factors and items most 
related to cognitive development, and the present study has 
confirmed those findings. In addition, this study has suggested 
that other factors or items which are much less strongly related 
to cognitive development may instead reflect environments conducive 
to the attainment of affective outcomes. These items provide 
further support for the validity of students' evaluations, and 
should, of necessity be included in evaluation questionnaires if 
coverage of both cognitive and affective development is desired. 
Certainly, uSing criteria related to both cognitive and affective 


outcomes, it is possible to obtain a much broader insight into 


teaching effectiveness. 
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TABLE ONE 


CORRELATIONS OF STUDENT COGNITIVE AND AFFECTIVE DEVELOPMENT WITH EOQ STUDENT 
RATINGS ON THE SEVEN EVALUATION FACTORS AND TWO SUMMARY RATING ITEMS 


Evaluation Factor/Item Correlation of EOQ Evaluations With-- 


Cognitive Affective Development Item 
Development 
32 33 34 35 36 

CONCERN -43* sak .36* .37* - 26 32 
ORGANIZATION -20 34 -39* .18 mp Be ef 
LEARNING 28 - 78%*% - 70%* - 70%* - 798* -62** 
INTERACTION -28 21 -28 ~36* -26 -36* 
BREADTH 24 28 -07 19 -00 -20 
DIFFICULTY oak -28 -05 wer oA5 24 
EXAMINATIONS -44* -42* -57** .38* -32 -38* 
OVERALL RATING OF 
INSTRUCTOR -41* - 26 -30 o31 -22 29 
OVERALL RATING OF 
COURSE 19 -63%* -63%* -59** .68%* 524% 


®P < .05, two-tailed. 
#ep < .01, two-tailed. 
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APPENDIX I 


Factor Analysis of the Evaluation Items 


A factor analysis (principal components solution followed 
by a varimax rotation) of the correlation matrix based on 
end-of-quarter evaluations produced the results shown below. 
The loading of each item on the factor it was designed to 
measure appears in a bold box; loadings of less than .20 
are indicated with dashes. Every item loads higher on its 
own factor than on any other. The factors are moderately 
intercorrelated, with correlations ranging from .03 to .50 
(median .25). The factors are generally positively related 
to each other, with the exception of the DIFFICULTY factor; 
it has low negative to zero correlations with the other factors. 
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Evaluation Items 


I LEARNING 


Intellectual curiosity in subject stimulated 
Learned something valuable 

Present interest in course subject 

Developed understanding of practical implications 


Degree of course mastery 
TI] CONCERN 
Instructors presentations made subiect understandable - - - - - - 
Instructor concerned with student learning/understanding - - .30 - - - 
Instructor enthusiastic about teaching - - 34 - - “' 
Instructor made course relevant - - - .40 cS = 
IIT ORGANIZATION 
Course material outlined and carefully explained - 20 «25 . . “ 
Course objectives stated and agreed with those actually 
pursued - 24 


Nature/purpose of assignments clear - - 
Presentation well prepared and integrated - 
Workload evenly spread over term - 
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IV STUDENT-TEACHING INTERACTION 


Students welcomed to seek help/advice - - - - - - 
Students encouraged to ask questions and were given answers - +32 - - - - 
Students free to disagree and/or express own ideas - - - 
Y BREADTH OF COVERAGE 
Instructor contrasted implications of theories - - - - 


Instructor presented background/origin of ideas/concepts - - - - 
Instructor discussed different points of view - - - - 


VI EXAMINATIONS/GRADING 


Graded materials adequately measured your knowledge - - - - - 81 - 
Graded materials measured content as emphasized in course - - - - - 74 - 
Grading was fair and objective - : . - - - 56 


VII WORKLOAD/DIFFICULTY 
Workload/pace was difficult - - ° - = é 
Course difficulty - - - - i i 
Mours/week outside of class - - - = = 
OVERALL INSTRUCTOR RATING - +52 - 26 - a ‘i 


ot) 
OVERALL COURSE RATING ad 51 : 20 - - : : 


